Methods and systems for data processing and their applications

ABSTRACT

A method ( 400 ) of processing signal outputs of a plurality of topologically distinct sensors in response to stimuli is described. The method comprises obtaining ( 402 ) a plurality of temporal sensor outputs in parallel. Thereafter, features are extracted ( 406 ), the features having dynamic behavior pattern. The extraction is performed in a topology consistent way by arithmetic processing in parallel of neighboring temporal sensor outputs. Furthermore, a quality of the extracted features is being determined.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to the field of data processing. Moreparticularly, the present invention relates to methods and systems fordata processing as well as to their application in computerised systemsin general, such as robotics and vehicles/automobiles in particular.

BACKGROUND OF THE INVENTION

Applications of robotics are widely spread. Robots often are complex andsophisticated systems requiring high computational power. Whereascurrent robots for automation are often very high performance andreliable machines, they still are hardly capable of sharing mosteveryday life tasks with humans, specially in terms of autonomousbehaviour and of user-friendly human-machine interfaces. Moreparticularly, they are hardly capable of manipulation or interactionwith unknown or unpredictable objects or living beings.

Examples of everyday life tasks that have received much attention formany years from both neuro-physiological and robotics communities aregrasp and lift tasks. Two aspects are crucial for a stable grasp: theability of the hardware and software system to avoid object slip and theability to control in real-time the grasping force. Neuro-physiologicalstudies allowed a deep understanding of these two aspects in humans,analysing grasping in its simplest configuration, i.e. in which anobject is grasped between the opposed thumb and index fingers andlifted. The role of skin mechanoreceptors during grasp, the mechanismsof motor coordination as well as the strategies used in humans to avoidobject slip were widely investigated. In particular, it was shown thatthe adaptation of the grip force (the grasping force) to the frictionbetween the skin and the object takes place in the first 0.1 s after theinitial contact. It furthermore has been found that, after lifting ofthe object, secondary adjustments of the force balance can occur inresponse to small short-lasting slips, revealed as vibrations (detectedmainly by Pacinian corpuscles).

The concept of incipient slippage was introduced to indicate themicro-vibrations in the peripheral regions of the contact area thatappear just before macroscopic slip occurs. The detection of such smallvibrations is one of the main methods used to assure grasp stability inrobotic grasping. One proposed solution was to use dynamic sensors (e.g.piezoelectric strips) to detect incipient slippage, another was based onmeasurement of the friction coefficient between the object and fingers.

Slip detection may be based on frequency content analysis on outputs ofa sensor array or e.g. on the combined use of strain sensitive sensorsand artificial neural networks. Another proposed solution is to use afuzzy-based controller: the inputs to the controller are relativevelocity and acceleration between object and fingers, as well as thefrequency content of the force distribution on the capacitive sensorarray obtained through fast Fourier transform (FFT) analysis, while itsoutput is the closing speeds of the fingers. The main drawbacks of thesemethods are their high computational requirements, which make themunsuitable for real-time hand control and rapid reactions if the numberof sensors is high.

Recently, incipient slip was detected through vision based analysis ofthe deformation of the fingertip. From the side of dextrous manipulationcontrol, it was proposed to use dynamic sensing (PVDF strips) to detecttactile events, such as onset or offset of contact between finger andobject, or micro-vibrations during the onset of a slip event, orexternal perturbations, associated with transitions between specificphases of the manipulation task, thus introducing an event-driven graspcontroller.

For an artificial tactile system to be useful not only in roboticgrasping but more in general for manipulation and human-robotinteraction purposes, the localization of the stimulus is a must, thusmaking a dense and diffused array of tactile sensors essential.

Whereas significant progress has been made in the last years in thefabrication and miniaturization of tactile sensors and sensors arrays,implementing these in robotics for manipulation and/or human—robotinteraction purposes and its corresponding challenges is still to alarge extent unsolved. Slip detection is only one example of problemswith sensors arrays that need to be addressed.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide good apparatus ormethods for processing signals such as sensor output data, as well as toprovide good computerized systems, such as robots, using such methodsand systems for processing signals. It is an advantage of embodimentsaccording to the present invention that data processing of sensor outputsignals can be performed efficiently, resulting in systems useful forhuman-robot interaction and/or machine perception purposes in real time.The system may be useful for active perception and/or passiveperception. Manipulation by a robot is one example of a process that canbe used in human-robot interaction. Manipulation information may betactile information.

The above objective is accomplished by a method and device according tothe present invention.

The present invention relates to a method of processing signal outputsof a plurality of topologically distinct sensors in response tomechanical stimuli, the method comprising obtaining a plurality oftopologically arranged temporal sensor outputs in parallel, extractingfeatures with a dynamic behaviour pattern by arithmetic processing inparallel of neighbouring temporal sensor outputs e.g. in a topologyconsistent way, and a quality of the extracted features beingdetermined. The mechanical sensors may be analog sensors. It is anadvantage that an efficient system is obtained. It is an advantage ofembodiments according to the present invention that systems can beobtained that are able to use information from a potentially very largenumber of sensing elements to provide in real-time manipulation, such ase.g. stable grasp, with minimal manipulation forces of unknown objects,e.g. having unknown shape, weight, friction and surface texture.

Obtaining a plurality of temporal sensor outputs in parallel maycomprise sampling a plurality of sensor outputs in time to generate thetemporal sensor outputs.

The temporal sensor outputs may be sensor outputs from mechanicalsensors.

Arithmetic processing may comprise processing said neighbouring temporalsensor outputs in the temporal domain. The arithmetic processing may beprocessing without Fourier Analysis. The latter results in a fastersystem.

The method may comprise arranging the extracted features in a topologymaintaining array or arrays. It is an advantage of embodiments accordingto the present invention that these provide a processor that is based onrecognition of significant manipulation events, such as e.g. significanttactile events or events wherein objects undergo a specific, e.g.sudden, movement. It is also an advantage of embodiments according tothe present invention that systems as described above can, at the sametime, be robust against failures of isolated or subsets of sensors.

The method may be adapted for processing said sensor outputs forimproving active or passive perception of events or features occurringin the environment.

The present invention also relates to the use of a method for processingas described above, in automotive industry, automotive systems,automotive sensor systems or automotive processing systems.

Said extracting and arranging may comprise recognising features insensor outputs and preserving their spatial localisation.

The method further may comprise spatially filtering or spatio-temporalfiltering of the array or arrays.

The spatial filtering or spatio-temporal filtering may be adapted forderiving actions to be taken by a controller for controlling a carriersubstrate of the plurality of sensors.

The spatially filtering may comprise extraction of an event based on aplurality of neighbouring extracted features in said topologymaintaining array or arrays.

The event may be manipulation by a computerised system, such as e.g. arobot, with a plurality of sensor outputs of an object, such as graspingof an object. The event may be the movement or the intention of amovement of an object in a field of view. More generally it may be anaction related to perception of the environment of the computerisedsystem. The neighbouring extracted features may be indicative of a blob,i.e. contact between a plurality of neighbouring sensors and the objector activation or increased sensing of a plurality of neighbouringsensors.

The arithmetic processing may comprise applying a bandpass filter or acombination of bandpass filters for temporal feature extraction.

The arithmetic processing may comprise using a cellular neural networkor a cellular non-linear network.

The cellular neural network or cellular non-linear network may benon-learning. It is an advantage of embodiments according to the presentinvention that Fourier transform analysis can be avoided, resulting inan efficient system, e.g. capable of processing a large amount ofsignals in parallel. It is an advantage of embodiments according to thepresent invention that they provide a parallel data processingarchitecture based on a cellular non-linear network (CNN) paradigm. Thelatter advantageously may allow design and implementation of brain-likeparallel algorithms.

The arithmetic processing may comprise only adding, subtracting,delaying, thresholding and logical functions. It is an advantage ofembodiments according to the present invention that Fourier transformanalysis can be avoided, resulting in an efficient system, e.g. capableof processing a large amount of signals in parallel.

The arithmetic processing may comprise applying a filter having a singlefilter input signal and resulting in multiple filter output signal andwherein each filter output signal is based on the subtraction of amodified input signal from an original input signal.

Arithmetic processing in parallel of neighbouring temporal sensoroutputs may comprise arithmetic processing in parallel of only part ofthe neighbouring temporal sensor outputs.

The number of neighbouring temporal sensor output data may be limited toless than 50% of the number of sensor outputs available. This may beuser-determined.

The feature with dynamic behaviour may comprise a feature indicating anyof a variation, vibration and/or oscillation. A variation thereby may bea change in intensity by at least a predetermine threshold in apredetermined period. An oscillation thereby may be a sequence of twovariations of opposite sign (and independent user-determined amplitude)in the same predetermined period. A vibration may be a sequence of atleast two oscillation in two subsequent periods of time.

The quality of the extracted features may comprise a stability of thedynamic behaviour among a plurality of neighbours. A stability may be aconsistency of occurrence of the dynamic behaviour pattern among aplurality of neighbours.

The present invention also relates to a method for controlling acomputerised system comprising at least one movable element comprising aplurality of topologically distinct sensors, the method comprisingmeasuring a plurality of topologically arranged output signals, e.g.from sensors which may be for example mechanical sensors or opticalsensors, processing said plurality of output signals using a method asdescribed above and controlling said at least one movable element basedon said determined quality of the extracted features.

The present invention furthermore relates to a data processor forprocessing signal outputs of a plurality of topologically distinctsensors in response to mechanical stimuli, the processor comprising aninput means for receiving a plurality of topologically arranged temporalsensor outputs in parallel, a feature extracting means for extractingfeatures with a dynamical behaviour pattern, the feature extractingmeans comprising an arithmetic processor for arithmetic processing inparallel neighbouring temporal samples of the sensor outputs e.g. in atopology consistent way, and a determination means for determining aquality of the extracted features. It is an advantage of embodimentsaccording to the present invention that these provide efficientinformation extraction, thus e.g. allowing the use of a large number ofsensors. It is an advantage of embodiments according to the presentinvention that control of computerised systems using a large number ofsensing elements, such as robots or e.g. more particularly robotichands, in real time operations, such as grasping and lifting, can beobtained.

The input means for receiving a plurality of temporal sensor outputs inparallel may comprise a sampling means for sampling a plurality ofsensor outputs in time to generate the temporal sensor outputs.

The arithmetic processor may be adapted for arithmetic processing ofsaid neighbouring temporal sensor outputs in the temporal domain.

The data processor furthermore may be adapted for arranging theextracted features in a topology maintaining array or arrays.

The data processor may be adapted for recognising features in sensoroutputs and preserving their spatial localisation.

The processor may comprise a filter for spatially or spatio-temporalfiltering of the array or arrays.

The filter may comprise filtering means for extraction of an event basedon a plurality of neighbouring extracted features in said topologymaintaining array or arrays.

The arithmetic processor may comprise a bandpass filter or a combinationof bandpass filters for extracting temporal features.

The arithmetic processor may comprise a cellular neural network or acellular non-linear network.

The arithmetic processor may comprise a filter based on onlysubtracting, delaying, thresholding, logical and/or morphologicalfunctions.

The arithmetic processor may comprise a filter having a single filterinput signal and resulting in multiple filter output signal and whereineach filter output signal is based on the subtraction of a modifiedinput signal from an original input signal.

The arithmetic processor for processing in parallel neighbouringtemporal sensor outputs may be an arithmetic processor for arithmeticprocessing in parallel only part of the neighbouring temporal sensoroutputs.

The dynamic behaviour may comprise any of a variation, vibration and/oroscillation.

The quality of the extracted features may comprise a stability of thedynamic behaviour among a plurality of neighbours.

The present invention also relates to a computerised system comprisingat least one movable element comprising a plurality of topologicallydistinct mechanical sensors, and a data processor as described above.

The system further may comprise a controller for controlling the atleast one moveable element as function of the obtained quality of theextracted features.

The present invention furthermore relates to a computer program productfor processing signal outputs of a plurality of topologically distinctmechanical sensors in response to mechanical stimuli, the computerprogram product, when executed on a computer, adapted for executing amethod as described above.

The present invention also relates to a machine-readable data storagedevice storing the computer program product as described above and/or tothe transmission of such a computer program product over a local or widearea telecommunications network.

It is an advantage of embodiments according to the present inventionthat systems with an appropriate sensory-motor coordination can beobtained.

It is an advantage of embodiments according to the present inventionthat repeated manipulation tasks can be performed in a stable andsuccessful way. For example, for repeated pick and lift tasks,embodiments of the present invention were able to guarantee stable graspand recognition of the onset of slippage with common objects likeplastic bottles, as well as with deformable objects like soft spongeballs and even extremely slippery, deformable, and delicate objects,such as Japanese tofu.

It is an advantage of some embodiments according to the presentinvention that data processors and computerised systems can be obtainedthat are able to deal with many analog signals, adjusting in real timethe manipulation force, e.g. grasp force, and keeping it at low, e.g.minimum, required values.

Particular and preferred aspects of the invention are set out in theaccompanying independent and dependent claims. Features from thedependent claims may be combined with features of the independent claimsand with features of other dependent claims as appropriate and notmerely as explicitly set out in the claims.

The teachings of the present invention permit, amongst others, thedesign of improved computerised systems such as for example robots orvehicles/automobiles and corresponding methods such as for examplemethods for providing human-robot interaction, methods for providingrobotic manipulation or methods for exploiting machine perception.

The above and other characteristics, features and advantages of thepresent invention will become apparent from the following detaileddescription, taken in conjunction with the accompanying drawings, whichillustrate, by way of example, the principles of the invention. Thisdescription is given for the sake of example only, without limiting thescope of the invention. The reference figures quoted below refer to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic representation of a data processor and relatedcomponents, according to an embodiment of the first aspect of thepresent invention.

FIG. 2 a and FIG. 2 b illustrate exemplary optical coded input data asmay be used in data processors according to embodiments of the presentinvention.

FIG. 3 is a schematic representation of a possible data flow in anexemplary data processor according to an embodiment of the presentinvention.

FIG. 4 is a schematic representation of the distribution of thedifferent processing tasks performed in an exemplary data processor andthe corresponding position where the processing is performed, accordingto an embodiment of the present invention.

FIG. 5 is a schematic representation of a compact filter for detectingvariations as can be used in a data processor according to a firstparticular embodiment of the present invention.

FIG. 6 a to FIG. 6 b is a schematic illustration of components andinteraction in a cellular neural network or cellular non-linear networkas can be used in embodiments of the present invention.

FIG. 7 is a schematic illustration of a the dataflow in a cellularneural network or cellular non-linear network as can be used inembodiments of the present invention.

FIG. 8 is a schematic illustration of a computerised system according toembodiments of the second aspect of the present invention.

FIG. 9 is a schematic representation of an exemplary method forprocessing sensor output signals according to embodiments of the presentinvention.

FIG. 10 illustrates a computer system that can be used as host computerfor performing a method according to embodiments of the third aspect ofthe present invention.

FIG. 11 illustrates a computerised system based on a biomechanical hand,being an example of a computerised system according to an embodiment ofthe second aspect of the present invention.

FIG. 12 a and FIG. 12 b illustrate definitions of features andparameters used in the extraction of features, as used in thedescription of an exemplary method for manipulating an object accordingto embodiments of an aspect of the present invention.

FIG. 13 shows a block diagram for an algorithm for detecting tactileevents being an illustration of a method for manipulating an objectaccording to embodiments of an aspect of the present invention

FIG. 14 shows an exemplary algorithm for detecting tactile signaloscillations as can be used in a method for manipulating an objectaccording to embodiments of an aspect of the present invention.

FIG. 15 shows an exemplary algorithm for checking the state of thesensors of a tactile system as can be used in a method for manipulatingan object according to embodiments of an aspect of the presentinvention.

FIG. 16 shows an exemplary algorithm for determining a stability of agrasp or lift as can be used in a method for manipulating an objectaccording to embodiments of an aspect of the present invention.

FIG. 17 shows a model of a method for grasping and lifting an objectbeing an example of a method for manipulating an object according toembodiments of an aspect of the present invention.

FIG. 18 shows an example of an event of a pedestrian crossing the streetwhich can be detected using a system and/or a method according toembodiments of an aspect of the present invention.

FIG. 19 shows an example of an event of a car moving in a street, whichcan be detected using a system and/or method according to embodiments ofan aspect of the present invention.

In the different figures, the same reference signs refer to the same oranalogous elements.

DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The present invention will be described with respect to particularembodiments and with reference to certain drawings but the invention isnot limited thereto but only by the claims. The drawings described areonly schematic and are non-limiting. In the drawings, the size of someof the elements may be exaggerated and not drawn on scale forillustrative purposes. The dimensions and the relative dimensions do notcorrespond to actual reductions to practice of the invention.

Furthermore, the terms first, second, third and the like in thedescription and in the claims, are used for distinguishing betweensimilar elements and not necessarily for describing a sequence, eithertemporally, spatially, in ranking or in any other manner. Moreover, theterms top, bottom, over, under and the like in the description are usedfor descriptive purposes and not necessarily for describing relativepositions. It is to be understood that the terms so used areinterchangeable under appropriate circumstances and that the embodimentsof the invention described herein are capable of operation in othersequences or orientations than described or illustrated herein.

It is to be noticed that the term “comprising”, used in the claims,should not be interpreted as being restricted to the means listedthereafter; it does not exclude other elements or steps. Similarly, itis to be noticed that the term “coupled”, should not be interpreted asbeing restricted to direct connections only. It means that there existsa path between an output of A and an input of B which may be a pathincluding other devices or means. “Coupled” may mean that two or moreelements are either in direct physical or electrical contact, or thattwo or more elements are not in direct contact with each other but yetstill co-operate or interact with each other.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure or characteristicdescribed in connection with the embodiment is included in at least oneembodiment of the present invention. Similarly it should be appreciatedthat in the description of exemplary embodiments of the invention,various features of the invention are sometimes grouped together in asingle embodiment, figure, or description thereof for the purpose ofstreamlining the disclosure and aiding in the understanding of one ormore of the various inventive aspects. Furthermore, while someembodiments described herein include some but not other featuresincluded in other embodiments, combinations of features of differentembodiments are meant to be within the scope of the invention, and formdifferent embodiments, as would be understood by those in the art.

In the description provided herein, numerous specific details are setforth. However, it is understood that embodiments of the invention maybe practiced without these specific details. In other instances,well-known methods, structures and techniques have not been shown indetail in order not to obscure an understanding of this description.

The terms “data processor” and “data processing means” may beinterchanged in the present application and both refer to the componentadapted for processing the data or signals. The term “processor” or“processing means” may be interchanged in the present application andboth refer to a component adapted for processing, e.g. signals. The term“input means” or “input data receiver” may be interchanged in thepresent application and thus both refer to the component adapted forreceiving the input signals. The terms “sampler” and “sampling means”,as well as “feature extractor” and “feature extracting means” and“quality determinator” and “quality determination means” respectivelymay be interchanged in the present application. Unless specificallystated otherwise, as apparent from the following discussions, it isappreciated that throughout the specification discussions utilizingterms such as “ascertaining,” “processing,” “computing,” “calculating,”“determining” or the like, refer to the action and/or processes of acomputer or computing system, or similar electronic computing device,that manipulate and/or transform data represented as physical, such aselectronic, quantities into other data similarly represented as physicalquantities.

The term “real-time” may be used for processes that proceed at a givenand predictable rate that matches that of the real world process. Inparticular it may refer to a detection of an event within apredetermined maximum latency corresponding with real world events antthe rate at which the computation proceeds may be high enough to assurequick response.

The invention will now be described by a detailed description of severalembodiments of the invention. It is clear that other embodiments of theinvention can be configured according to the knowledge of personsskilled in the art without departing from the true spirit or technicalteaching of the invention, the invention being limited only by the termsof the appended claims.

In a first aspect, the present invention relates to a data processor forprocessing outputs of a plurality of topologically distinct sensors,such as for example mechanical sensors, e.g. pressure, force or acousticsensors; electromagnetic sensors such as e.g. optical sensors, magneticsensors, etc. The data processor may be particularly suitable forsensors providing a sensor output as response to mechanical stimuli.Mechanical stimuli thereby may be movement of an object, a change inpressure by an object, etc. Such mechanical stimuli may, in agreementwith the sensors indicated above, be sensed in an optical way, in a wayusing pressure or force, in an acoustic way, using electromagneticinteraction, using magnetic interaction, etc. In some embodiments, theobject does not need to be or come in direct physical contact with thesensor in order to sense the mechanical stimuli. In other advantageousembodiments, the object is or comes in direct physical contact with thesensor or sensing surface. The data processor may be suitable for use inor with computerised systems, such as e.g. robots, computerisedprosthetic devices such as a robotic arm or leg, machinery for use inmanufacturing environments, but also in other computerised systems suchas e.g. in transportation means such as automobiles, vehicles, . . . .Such computerised systems may comprises a plurality of sensors allowingmachine perception and/or control of, interaction with and/ormanipulation of objects, e.g. unknown objects, making such computerisedsystems suitable for e.g. everyday tasks and/or system—humaninteraction.

By way of illustration, the present invention not limited thereby,standard and optional components of an exemplary data processor 100 areshown in FIG. 1. The data processor 100 may receive input signals from acomputerised system 10, which may be a peripheral system, comprising aplurality of sensors 20, such as for example mechanical sensors, e.g.pressure, force or acoustic sensors electromagnetic sensors such as e.g.optical sensors, magnetic sensors, etc. The computerised system 10,which may be external to the data processor or in which the dataprocessor may be integrated, may be controlled using a controller 30,adapted for generating control signals. Embodiments according to thepresent invention include or can be used, for example, in computerisedsystems, such as for example robotics for human-robot interaction,manipulation and/or control, humanoid robotics, as well as prostheticdevices, machinery for manufacturing purposes, but also for example incomputerised systems in transportation means such as vehicles e.g.automobiles. Embodiments of the present invention may be especiallysuitable for computerised systems adapted for performing tactile events,such as lifting or grasping of objects or living beings. More generally,embodiments of the present invention may provide higher levels ofmachine perception or understanding, e.g. of the environment of themachine.

The data processor 100 comprises an input means 102, e.g. also referredto as data receiver, for receiving a plurality of sensor outputs.Preferably the plurality of sensor outputs, being input data for thedata processor 100, may be obtained in parallel. The received sensoroutputs may be any type of data or signals, such as e.g. electricalsignals, optical signals, etc. An advantage of optical coded input isthat it allows efficient receiving of data in parallel. These inputsignals may for example be one or more electrical sensor outputs persensor that have been processed and converted into optical coded signalssuch as light pulses. In one example, the input means 102 may forexample comprise an optical receiver, such as e.g. a CCD or C-MOSoptical input layer, for receiving optical signals. The input signalsthus may be obtained as optical signals, e.g. transferred via opticalfibres such as plastic optical fibres, although the invention is notlimited thereto. The input means 102 may be adapted for receiving theplurality of sensor outputs as optical signals, e.g. as an optical imagewhereby each pixel of the image could ideally correspond to one sensorsignal on the computerised system, e.g. hand. Nevertheless, theimplemented algorithms for controlling the system are resolutioninvariant such that also groups of pixels of the image may berepresentative of individual sensor outputs.

The number of input data or signals that may be obtained in parallel maybe large, e.g. it may be more than 2, e.g. more than 5, or more than 8,or more than 16, or more than 32, or more than 50, or more than 100 orthousands. It is an advantage of embodiments according to the presentinvention that a large number of sensor outputs can be obtainedsimultaneously as this will assist in efficient processing of thesignals. Furthermore, the latter will assist in detecting isolatedmalfunctioning detectors or detector arrays.

The input data may be received in a topologically consistent way, i.e.the relative geometrical positions of the received data on the inputmeans 102 may replicate the relative position of the sensors thatoriginate them. In some embodiments, the present invention not beinglimited thereto, there may be a homeomorphism between the sensor spaceand the pixel space. Two sensors close to each other in reality may havetheir projections onto the input means 102 close to each other. In someembodiments, if different types of sensors are used, the input data foreach type of sensors may be received in a topologically consistent way.In some embodiments, a plurality of signal outputs may be extracted byeach sensor and the geometry of sensor channels on the input means 102may be chosen accordingly to the processing foreseen.

In some exemplary embodiments, the positions of the sensors and thepositions of the received optical input, e.g. the position of the pixelswhen the input is received as image, advantageously are topologicallyconsistent.

In one particular example the input means 102 is a 128×128 C-MOS opticalinput layer of a visual microprocessor, such as for example an ACE16KCNN visual microprocessor, used in an industrial PC. By way ofillustration, the present invention not being limited thereto, twoexamples of input data as obtained in such a system are represented inFIG. 2 a and FIG. 2 b. The input data shown in these drawings illustrateinput data for a robotic arm whereby 54 tactile optical signals aremapped on the 128×128 C-MOS optical input layer, for two differenttactile images.

A Cellular non-linear Network (CNN), (or cellular neural network), is an-dimensional array of identical dynamical systems (called cells) thatinteract directly within a finite local neighbourhood, and whose statevariables are continuous valued signals. CNNs are a computation paradigmfor problems reformulated as a well-defined task where the signal valuesare placed on a regular 2-D grid (e.g. image or array processing). LocalCNN connectivity allows implementation as a VLSI (as in the ACE16K CNNvisual microprocessor) or FPGA which can perform image processing basedon local operations at a very high speed.

The data processor 100 furthermore may comprise a sampling means 104 forsampling the plurality of sensor outputs in time. In this way, temporalsamples are generated. The sampling means 104 may be integrated to orseparated from the input means 102. If e.g. the sensor output signals,which are the input signals for the data processor, are optically coded,the sampling may be performed by setting an integration time on theinput means, then being an optical receiver. The sampling may e.g. beperformed by controlling a shutter in an optical receiver. In this way,the obtained sensor data will become grey scale images. Nevertheless,other types of sampling, e.g. on other types of signals, also may beused. Sampling also may be performed at the peripheral part of thesystem, so that sampled signal outputs are received in the dataprocessor. In some embodiments of the present invention, sampling may beperformed both at the peripheral part of the system and in the dataprocessor. For example, at the peripheral level tactile signals may besampled in the skin modules and converted into light pulses and the dataprocessor provides a second sampling. The data processor may be directlyfed with analog signals, it does not need to convert into digitalsignals.

The data processor may comprise a shutter for appropriately filling orcharging cells of the data processor with data signals. When opticalsignals are used for filling or charging cells of the data processor, aswill be discussed in more detail below, the optical signalsadvantageously may be analog light signals, resulting in an more easyhandling of the data by the data processor, compared to filling orcharging using digital light signals, e.g. modulated by pulse widthmodulation.

The data processor 100 furthermore comprises a feature extracting means106 for extracting features with a dynamical behaviour pattern. Thedynamical behaviour pattern may be a dynamic behaviour that is specificfor the manipulation task performed by the computerised system. Thefeatures may be any type of features that are characteristic for anevent to occur during the manipulation. Such events may be for exampleslip, contact such as grasping or hitting, release of an object,movement of an object, rotation of an object, variation in a movement ofan object, etc. The features characteristic for such events may be achange in the sensor signals, e.g. larger than a certain threshold. Sucha change may be indicated as a variation. Other features may be thechange of a sensor signal in a predetermined way, such as e.g. amodulation of the signal, (monotone) increase or decrease, a changefulfilling a predetermined function, etc. Other features that may beused can be oscillation or vibration, e.g. as described in more detailin one of the examples. By way of example, the present invention notbeing limited thereto, for the task of grasping and lifting dynamicbehaviour pattern may be used referred to as variation, oscillation orvibration. For the task of detecting sudden movements of pedestrians orvehicles in traffic, variation, oscillation or vibration of parts of theobject or of the full object can also be used. In addition to thefeatures having a dynamic behaviour pattern, other features also may beused, such as for example area over which the sensor activation isspread, average intensity of the sensor signals, spreading of the sensoractivation, etc.

The extraction of features with a dynamic behaviour pattern is performedby arithmetic processing in parallel of neighbouring temporal samples ofthe sensor outputs, e.g. in a topology consistent way. The featureextracting means 106 may be adapted for arithmetic processing of theneighbouring temporal samples in the temporal domain. The features withdynamic behaviour pattern advantageously may be extracted taking intoaccount topology of the parallel processed samples of the sensoroutputs. It may be adapted for processing without using Fouriertransform analysis. Processing tasks of the feature extracting means 106may be performed by an arithmetic processor 110, that may be dedicatedto the feature extracting means 106 or commonly used by differentcomponents of the data processor 100. Different types of arithmeticprocessors may be used. For example in one embodiment the featureextracting means 106 may comprise an arithmetic processor 110 acting asor using a bandpass filter or a plurality of bandpass filters or afilter providing the function of a plurality of bandpass filters withdifferent bands for temporal feature extraction. The feature extractingmeans 106 may comprise an arithmetic processor 110 comprising a cellularneural network or cellular non-linear network (CNN) as will be describedin more detail below. The feature extracting means 106 may be adaptedfor acquiring all sensory input in parallel and process it in parallel.It furthermore may be adapted for processing these sensory input inparallel in the analog domain. The latter may be performed at highspeed. The feature extracting means 106 may be adapted for extractingfrom the input stimulus features similar to those extracted by the humanmanipulation system using biologically inspired algorithms. The lattercan be implemented in a computationally efficient way. The featureextracting means 106 furthermore may be adapted for taking into accountfunctional aspects of the human sensory-motor coordination strategies.It is an advantage of embodiments according to the present inventionthat an efficient and robust biologically-inspired control algorithm isobtained. The method may also comprise extraction of features with astatic behaviour pattern performed by arithmetic processing in parallelof neighbouring temporal samples of the sensor outputs.

The data processor 100 also comprises a quality determination means 108for determining a quality of the extracted features. The quality of theextracted features may be for example a stability of the dynamicbehaviour, an intensity of the dynamic behaviour, a consistency of thedynamic behaviour among a plurality of neighbouring data or acombination thereof. The stability thereby may be defined as theconsistency of occurrence of the dynamic behaviour pattern over aplurality of time. These quality aspects may be evaluated with respectto a predetermined threshold. The data processor 100 thus may comprisesprocessing capacity for determining a quality of the extracted features.The latter preferably may be performed by the same arithmetic processor110 as the arithmetic processor used for feature extraction (as shown inFIG. 1), although the invention is not limited thereto, and a differentarithmetic processor can be used. The determined quality of theextracted features may be used by an internal or external controller forcontrolling the computerised system. The determined quality of theextracted features also may be used to generate semantic informationregarding the manipulation, whereby the semantic information is used byan internal or external controller for controlling the computerisedsystem. The semantic information relates to information regarding themanipulation occurring in the real word. Semantic information maycomprise information regarding the manipulation, such as e.g. slip,stable contact, a sudden tactile event, a change in direction ofmovement, a movement, a sudden movement, etc. The semantic informationmay be generated from the quality data using the arithmetic processor orusing any other suitable processor. The generation of semanticinformation may also partly be based on proprioceptive information fromthe controller, e.g. resulting in higher level semantic information.Such proprioceptive information may be data generated by thecomputerised system and may be for example related to its jointsposition, like for example, when the computerised system is a roboticarm, the degree of closure of a hand and/or the height of the arm.

By way of illustration, the present invention not being restrictedthereby, an exemplary dataflow for the data received, handled andprocessed by the data processor 100 is shown in FIG. 3. The dataprocessor 100 receives sensor output data 202, that may be pre-processedand/or converted signals. Such data is received from the peripheralsystem, e.g. from the sensors from the computerised system. Such signalsmay e.g. be optical signals representing tactile signals that arepre-processed and electro-optically converted. The sensor output data202, which are input data for the data processor 100, are temporal data204, optionally obtained by sampling of the input data for the dataprocessor 100. Another example is that the input data is temporal data204 obtained as continuous signals. The temporal data then arearithmetically processed resulting in a set of extracted features 206.The extracted features may be arranged in a topology maintaining arrayor arrays and extraction of features may therefore be performed takinginto account topology of the sensor outputs. From the extracted featuresa quality of the extracted features 208 is obtained, corresponding withsemantic information regarding the manipulation. An example of suchsemantic information may be a combination of a subset of features beingindicative of slippage when contact was first established, as will beillustrated below. The quality of extracted features 208 orcorresponding semantic information is then outputted to the controllerfor controlling the computerised system, where it may be transferred incontrol information 210. Such a controller may be internal to the dataprocessor 100 or external to the data processor 100. By way of example,in FIG. 1, the controller is indicated as external to the data processor100. The controller may combine the obtained information withproprioceptive information such as e.g. position signals from thecomputerised system. In this way, higher level semantic information maybe generated. For example for a robotic hand, if slip is detected andcontact was established, but in practice the hand is open, this isindicative of something being wrong. The different processing hardwarelevels at which the data is generated, handled and used are indicated inFIG. 4. The input data 202 are generated at the sensor modules, i.e.part of the computerised system 10, where pre-processing may beperformed. Also electro-optical conversion of the sensor output may beperformed, the latter depending on the type of signals that can bereceived by the data processor. Once received and optionallyappropriately sampled by a sampler, the arithmetic processor processesthe input date for extracting features and for determining a qualitythereof. The arithmetic processor may be positioned in the dataprocessor 100. Semantic data generated in the arithmetic processor maycomprise the determined quality of the extracted features or comprise aninterpretation of the quality. The quality data or semantic data then istransferred (outputted) to the controller processor, positioned in thecontroller 30, and may be processed into control data for controllingthe computerised system.

In embodiments of the present invention, the data processor 100 may beadapted for storing the arithmetic processed signals in a topologicallyconsistent way. The latter may e.g. be in a topological consistentarray. In other words, the topology of the sensors on the computerisedsystem 10 may not only correspond with the topology of the input datafor the data processor 100, it further also may correspond with thetopology of the processed signals after arithmetic processing of thesignals. In other words, the extracted features may be arranged in atopologically consistent way corresponding with the topology of thesensors from which the input sensors are obtained. The extractedfeatures may for example be stored in a topologically consistent arrayor arrays, topologically consistent with a position of the sensors usedas input for the data processor. Topology may be taken into account,e.g. maintained, for the extraction of features having a dynamicbehaviour pattern.

In embodiments of the present invention, the data processor 100 may beadapted for, besides extracting features with a dynamic behaviourpattern, also extract features with a spatial behaviour or with aparticular intensity behaviour. In other words, the data processor 100additionally may be adapted for obtaining spatial features like area anddistribution of the sensors activated during the manipulation orintensity features like the average intensity of the activatedsensor(s). These features may be, optionally after being processed, usedtogether with the extracted features having a dynamic behaviour patternfor determining the quality of the extracted features or, for example,may be used for together with the quality of the extracted features,determining the semantic situation. For example, besides the extractionof features with a dynamic behaviour pattern by handling the sensorsignals in parallel in the temporal domain, spatial filtering may beperformed for extracting spatial information.

The data processor 100 may be adapted for determining or detectingmanipulation events as well as detecting a quality of a feature orcorresponding event, such as for example contact between thecomputerised system 10 and the object. The data processor 100 moregenerally may be adapted for determining or detecting perception events,e.g. with respect to environmental features of the computerised system.

Particular embodiments according to the first aspect of the presentinvention will be further described in more detail, the presentinvention not being limited thereby, but only by the claims.

In a first particular embodiment, the present invention relates to adata processor 100 for processing signal outputs as described above,whereby at least the feature extracting means 106 is adapted with anarithmetic processor 110 for filtering the temporal samples of the inputdata using a bandpass filter or a combination of a number of bandpassfilter or a filter providing the functionality of a combination of anumber of bandpass filters with distinct bands that only operate in thetemporal domain. The bandpass-filter(s) may be a spatial-temporalbandpass filter(s) only operating in the temporal domain. It may operatewithout Fourier transform. The bandpass filter(s) may be applied to eachindividual data signal received at each step in parallel. It maycomprise a feedforward COMB filter. Contrary to a conventional combfilter, the signals may be subtracted. The bandpass filter(s) maycomprise a temporal COMB filter, wherein at each time the output signalis a combination, e.g. subtraction, of only two signals, not of allprevious samples in the window. The bandpass filter may be non-linear.It may comprise blocks of non linear functions. It may comprise logicaloperators and/or it may comprise morphological operators for denoisingor other operations on the input signals. The bandpass filter(s) maycomprise a combination of any or all of a feedforward COMB filter,blocks of non linear functions, logical operators and morphologicaloperators for denoising. The bandpass filter(s) may comprise onlyfunctionality of subtracting, delaying, thresholding and applyinglogical functions. The latter limits the required computational power,and therefore advantageously allows a quicker operation time. Thebandpass filter(s) may be single input, multiple output and each filteroutput signal may be based on the subtraction of a modified input signalfrom an original input signal. By way of illustration, the invention notlimited thereby, an explicit structure of a filter that can be used isshown in FIG. 5. The filter structure includes a feedforward COMBfilter, the non-linear functions and logical or functions. Filteroperation further more includes following features: x_(i)[n], y_(i)[n]are time signals in one example corresponding with pixel grey values.The filter operation thus can be applied on the whole data, e.g. image,in parallel, i.e. to all of the individual input data in parallel. Thesignals x_(i)[n], y_(i)[n] in one example thus can be considered asimages. The filter operates in tiling time windows having a lengthcorresponding with a number of samples N, the tiled windows beinggenerated by increasing the delay k from 1 to N. The number of samples Nmay be selected as function of the sampling rate, the required bandpassand required latency of the filter. During filter operation the delay kthus increases at each iteration, and the output signals y_(i)[n] areequal to OR{f_(i)(x[n]−x[n−1]), f_(i)(x[n]−x[n−2]), f_(i)(x[n]−x[n−k])},where f_(i)(x) are threshold functions. At the end of the cycle, whenk=N, following actions may be performed: y₁[n] . . . y₈[n] are combinedby a series of logical functions (AND, OR, NOT), to isolate the searchedfeatures, y₁ . . . y₈ are reset and k is set equal to 1. In other words,after each N iterations, the M output signals are combined in a seriesof logical operations to extract features.

In a second particular embodiment, the present invention relates to adata processor 100 for processing signal outputs as described above,e.g. as described in the first particular embodiment but not limitedthereto, whereby at least the feature extracting means 106 is adaptedwith an arithmetic processor 110 comprising or consisting of a cellularnon-linear network (CNN). The latter may be particularly suited forprocessing analog array signals. The cellular network may be hardwareimplemented, software implemented or a combination of hardware andsoftware implemented. The cellular network may e.g. be hardwareimplemented on an integrated circuit chip. The latter may be a verylarge scale integration (VLSI) chip, the invention not being limitedthereto. The CNN architecture may be a two-dimensional array of locallyconnected analog dynamic processors or cells. Such cellular network maybe a non-learning network. In one example, the cellular network may be acellular network architecture coupled to an optical input layer, such ase.g. a 128×128 cells cellular network architecture of the ACE16K CNNvisual microprocessor as described above. In this case each pixel of anoptical receiver may correspond with a cell of the cellular networkarchitecture. By using a visual microprocessor with CNN processor, theCNN processor may accept and process images as data input in paralleland on a continuous time basis. Nevertheless, more generally thecellular network may receive its input from the input means 102, afterthe input signals have optionally been sampled by the sampling means104. It is an advantage of embodiments according to the presentinvention, that the CNN architecture mimics to some extent the anatomyand physiology of many biological sensory and processing systems. Theapplication of methods and systems according to the present inventionallows that brain-like information processing algorithms, intrinsicallyanalog and spatio-temporal, can be implemented. The latter results in atleast some of the advantages as set out in the present description.

The CNN may be a group of locally connected analog dynamic processors,or cells. By way of illustration, the present invention not beinglimited thereto, a CNN being a two-dimensional array of locallyconnected analog dynamic processors is shown in FIG. 6 a and discussedin general. With reference to this figure, the cell in row i and columnj will be indicated by the notation C(i,j). FIG. 6 b illustrates thedifferent synaptic connections linking cell C(i,j) with the surroundingcells. The subset of cells directly connected with C(i,j) is indicatedby the notation S(i,j) or more particulary S_(r)(i,j), whereby rindicates the radius of influence for the cell C(i,j). By way ofillustration, in FIG. 6 b the situation for r being the interdistancebetween 2 neighbouring cells is shown. Each cell C(i,j) may becharacterised by a state x_(ij) and produces an output y_(ij). The statex_(ij) depends on its previous state, a combination of the inputs u toS(i,j), a combination of the outputs y of the previous computation onS(i,j), and an offset value z_(ij); y_(ij) is a non-linear function ofx_(ij), usually a saturation function. In a more formal notation, theequation determining the future state of a cell is

${\overset{.}{x}}_{ij} = {{- x_{ij}} + {\sum\limits_{{C{({k,l})}} \in {S_{r}{({i,j})}}}{{A\left( {i,{j;k},l} \right)}y_{kl}}} + {\sum\limits_{{C{({k,l})}} \in {S_{r}{({i,j})}}}{{B\left( {i,{j;k},l} \right)}u_{kl}}} + z_{ij}}$

while the output is related to the state by the non linear equation:

y _(ij) =f(x _(ij))

where A and B are two matrices, which may be referred to as feedback andinput synaptic operators respectively, and z_(ij) is the threshold ofC(i, j). The data input and data output for the CNN cells is shown inFIG. 7. The triplet [A,B, z], which may be referred to as cloningtemplate or simply template, completely characterizes each CNN. Oneexample of a CNN hardware implementation may be the use of two 3×3 realmatrices B, A (describing the weights of the connections between C(i, j)and its surrounding S_(r)(i, j)) and a real scalar z. It is an advantageof this embodiment that the CNN algorithm may comprise or consist of anordered sequence of templates, that are applied in parallel at the sametime to every input signal, topologically organized as a two dimensionalimage. It is an advantage of CNNs that they provide a suitablearchitecture to efficiently process large amounts of sensorialinformation. The CNN-based processor may use combinations of hand-shapefeatures as well as spatial and temporal features extracted from arraysof sensors to control in real-time the computerised system.

In a second aspect, the present invention relates to a computerisedsystem, such as e.g. a robot like a humanoid robot or a computerisedprosthesis. The computerised system comprises at least one moveableelement having a plurality of topologically distinct sensors providedthereon. The sensors may be sensors for example mechanical sensors, e.g.pressure, force or acoustic sensors; electromagnetic sensors such ase.g. optical sensors, magnetic sensors, etc. The data processor may beparticularly suitable for sensors providing a sensor output as responseto mechanical stimuli. The system therefore is a multi-sensor system.Mechanical stimuli thereby may be movement of an object, a change inpressure by an object, etc. Such mechanical stimuli may, in agreementwith the sensors indicated above, be sensed in an optical way, in a wayusing pressure or force, in an acoustic way, using electromagneticinteraction, using magnetic interaction, etc. In some embodiments, theobject does not need to be or come in direct physical contact with thesensor in order to sense the mechanical stimuli. In other advantageousembodiments, the object is or comes in direct physical contact with thesensor or sensing surface. A moveable element may be adapted forperforming manipulation of objects or for interacting with livingbeings. It may be actuated with an actuator. In particular embodimentsaccording to the present invention, the computerised system may be arobotic hand or arm with hand for manipulating objects and/orinteraction with living beings. Manipulation and/or interaction may bebased on tactile events. The sensors therefore may be used forregistering of tactile events, the invention not being limited thereto.More particularly, the system also may be used for interactions withobjects wherein no direct physical contact is present. The computerisedsystem may be adapted for everyday life tasks, such as e.g. grasp andlift tasks, tasks relating to displacement of objects, tasks relating toorientation of objects, etc. the invention not being limited thereto.The computerised system comprises a data processor as described in thefirst aspect of the present invention, comprising the same features andadvantages as set out in the first aspect. The data processor receivesits data from the sensors, e.g. after it has been pre-processed and/orconverted. The computerised system furthermore may comprise a controllerfor controlling the moveable element as function of the signal output ofthe data processor. The controller therefore is adapted for receivingoutput data from the data processor and processing this data to controldata for controlling the computerised system, e.g. the moveable elementof the computerised system. The controller may provide processingfunctionality that may be implemented in the same hardware environment,e.g. host computer like a personal computer, as the data processing,although the latter also may be implemented to a separate computingdevice in communication with the data processor. The processing capacityof the controller may be provided by any suitable type of processor inthe host computer such as for example a digital data processor or aprogrammable digital logic device such as a Programmable Array Logic(PAL), a Programmable Logic Array, a Programmable Gate Array, etc. Byway of illustration, the present invention not being limited thereto, anexample of a computerised system 300 is shown in FIG. 8, illustratingstandard and optional components of such a system. The computerisedsystem 300 comprises at least one moveable element 302, in the presentexample actuated by an actuator 304 which is controlled by a controller30. The moveable element 302 comprises a plurality of sensors 20, whichmay e.g. be implemented as skin mechanoreceptors also referred to assensitive skin receptors, the invention not being limited thereto. Themoveable element 302 furthermore may be equipped with a set of encodersproviding proprioceptive information, such as e.g. position informationof the moveable element, which may be used in the controller 30 fordetermining further control signals for controlling the moveable element302 in combination with the output of the data processor 100, that isadapted for receiving, processing and outputting processed data from thesensors. Further features and advantages of the computerised systems aredescribed and/or illustrated by the features of the data processor 100described in the first aspect as well as in the examples provided in thepresent invention.

In a third aspect, the present invention relates to a method forprocessing signal outputs of a plurality of topologically distinctsensors, such as for example mechanical sensors, e.g. pressure, force oracoustic sensors; electromagnetic sensors such as e.g. optical sensors,magnetic sensors, etc. The data processor may be particularly suitablefor sensors providing a sensor output as response to mechanical stimuli.Mechanical stimuli thereby may be movement of an object, a change inpressure by an object, etc. Such mechanical stimuli may, in agreementwith the sensors indicated above, be sensed in an optical way, in a wayusing pressure or force, in an acoustic way, using electromagneticinteraction, using magnetic interaction, etc. In some embodiments, theobject does not need to be or come in direct physical contact with thesensor in order to sense the mechanical stimuli. In other advantageousembodiments, the object is or comes in direct physical contact with thesensor or sensing surface. Such method can advantageously be used forcontrolling a computerised system, e.g. during or for manipulation ofobjects or interaction with living beings. The method comprisesobtaining a plurality of the sensor outputs in parallel and sampling theplurality of sensor outputs in time to generate temporal samples. Themethod furthermore comprises extracting features with a dynamicbehaviour pattern by arithmetic processing in parallel of neighbouringtemporal samples of the sensor outputs, e.g. in a topology consistentway. The latter may be performed taking into account or based on atopological arrangement of the sensor outputs. The method also maycomprise extracting features with a static behaviour pattern byarithmetic processing in parallel of neighbouring temporal samples ofthe sensor outputs. The method furthermore comprises determining aquality of the extracted features. By way of illustration, an exemplarymethod 400 for processing signal outputs of a plurality of distinctsensors is shown in FIG. 9, indicating standard and optional steps. In afirst step 402, the output signals of the plurality of sensors isreceived as input. The signals thereby are obtained in parallel, thusassisting in efficient processing. Receiving the input signals may be byoptically detecting the input signals, although other type of signals,such as e.g. electrical signals, also could be processed. The inputsignals may be analog input signals. In a second, optional step 404, theinput signals optionally may be sampled so as to obtain temporal data.Another example is that the input data received may be directly receivedas temporal data. The latter may be performed in any suitable way, suchas e.g. using a shutter in an optical receiving input unit. In a thirdstep 406, features having a dynamic behaviour pattern are extractedusing arithmetic processing. The latter may be performed taking intoaccount topology of the sensor outputs. The arithmetic processing isperformed in parallel on neighbouring temporal samples of the sensoroutputs, also referred to as signal input for the data processingsystem. Such arithmetic processing may comprise or consist of applying abandpass high-pass or low-pass filter or a combination of bandpass,high-pass or low-pass filters operating on the data only in the temporaldomain. It thus may comprise a filter sensitive to one or more bandsthat are not consecutive. The arithmetic processing may comprise orconsist of applying a filter wherein only thresholding, delaying,subtracting, logical and/or morphological functions are used. Thefiltering may be non-linear. The arithmetic processing may be performedwith or using a cellular non-linear network.

The processing may be performed on data organised as images. Thefeatures may be as described in the first aspect. In an optional fourthstep 408, the extracted features may be stored in a topologicallyconsistent format, e.g. in a topologically consistent array. In otherwords, the position of the sensor with respect to its neighbours in thecomputerised system may correspond with the stored position of theprocessed data of the sensor with respect to the stored processed dataof the neighbouring sensors. In an optional fifth step 410, additionalfeatures may be extracted by spatial filtering or a combination ofspatial and temporal filtering. The latter may be performed with anysuitable filter and may be used for extracting spatial features such asarea and distribution of sensor signals of activated sensors. In a sixthstep 412, a quality of the extracted features is determined. The lattermay be performed prior or after the optional spatial filtering. Qualitymay be as described in the first aspect. Determining of a quality maye.g. be determining of a stability of an extracted feature or acorresponding event. In an optional step 414, the determined qualitydata may be converted to semantic information, expressing the actualaction performed during the manipulation in the real world, and allowinga controller to further control the computerised system accordingly.Either this semantic information or the quality information may beprovided as output information to the controller. Other features andaspects of the method for processing may be features expressed by thefunctionality of the features of the data processor as described in thefirst aspect. In one embodiment, a further optional step may beperformed, advantageously at the start of the procedure. This furtheroptional step comprises tuning parameters in the proposed algorithm tothe current hardware configuration, e.g. determined by the materialsused, the sensors used, a thickness of the covering layer of thesensors, etc. The latter may e.g. be especially useful for manipulationwherein slipping may play a significant role.

In a fourth aspect, the present invention relates to a method ofcontrolling a computerised system. The computerised system therebycomprises at least one moveable element comprising a plurality oftopologically distinct sensors, such as for example mechanical sensors,e.g. pressure, force or acoustic sensors; electromagnetic sensors suchas e.g. optical sensors, magnetic sensors, etc. The method comprisesmeasuring a plurality of output signals from the sensors and processingsignal outputs using a method as described in the third aspect so as toobtain a quality of extracted features. The method furthermore maycomprise determining a status of the computerised system and/orcontrolling the at least one moveable element based on the quality ofextracted features. Such controlling may for example be controlling aposition or movement of the at least one moveable element. Such a methodmay assist in improving of perception of events occurring and featuresbeing present in the environment by the computerised system. Thefeatures and advantages of the method steps for processing the signaloutputs are the same as described in the third aspect and therefore notrepeated here.

In one embodiment, the method may comprise, prior to obtaining inputdata in the data processor, opto-electronically converting the electricor electronic signals of the sensors into optical signals so as toobtain the output signals of the sensors in an optically coded manner.The signals may be provided in parallel e.g. as image.

In one embodiment, the method for controlling a computerised system mayinclude an initial test step for checking whether the sensors areproperly operating. The latter may for example be performed bydetermining over a period of time the output signals of the sensors whennot triggered or actuated and deriving there from sensors that arenot-trustable, e.g. broken, so as to exclude these from use. The latteris especially useful when the plurality of sensors is redundant.

Controlling of the at least one moveable element may comprise firstconverting the information regarding quality of extracted features to asemantic data, representative of the manipulation situation in the realworld, and thereafter using the semantic data. Furthermore, controllingof the at least one moveable element may comprise taking into accountproprioceptive information, e.g. obtained from encoders on thecomputerised system. Other optional features of the method compriseprocess steps as induced by components of the computerised systemdescribed in the above aspects and the additional examples.

The above-described method embodiments of the third or fourth aspect ofthe present invention may be implemented in a processing system orprocessor as described in the first aspect. FIG. 10 shows oneconfiguration of processing system 500 that includes at least oneprogrammable processor 503 coupled to a memory subsystem 505 thatincludes at least one form of memory, e.g., RAM, ROM, and so forth. Itis to be noted that the processor 503 or processors may be a generalpurpose, or a special purpose processor, and may be for inclusion in adevice, e.g., a chip that has other components that perform otherfunctions, as long as they allow to provide the functionality expressedin the method steps of the claimed and/or described method. Thus, one ormore aspects of the present invention can be implemented in electroniccircuitry, or in computer hardware, firmware, software, or incombinations of them. The processing system may include a storagesubsystem 507 that has at least one disk drive and/or CD-ROM driveand/or DVD drive. The storage subsystem 507 may comprise a solid statememory for storing a computer program product for performing the stepsof the method embodiments as described above. In some implementations, adisplay system, a keyboard, and a pointing device may be included aspart of a user interface subsystem 509 to provide for a user to manuallyinput information. Ports for inputting and outputting data also may beincluded. More elements such as network connections, interfaces tovarious devices, and so forth, may be included, but are not illustratedin FIG. 10. The various elements of the processing system 500 may becoupled in various ways, including via a bus subsystem 513 shown in FIG.10 for simplicity as a single bus, but will be understood to those inthe art to include a system of at least one bus. The memory of thememory subsystem 505 may at some time hold part or all (in either caseshown as 511) of a set of instructions that when executed on theprocessing system 500 implement the steps of the method embodimentsdescribed herein. Thus, while a processing system 500 such as shown inFIG. 10 is prior art, a system that includes the instructions toimplement aspects of the methods for processing signals or data is notprior art, and therefore FIG. 10 is not labelled as prior art.

The present invention also includes a computer program product whichprovides the functionality of any of the methods according to thepresent invention when executed on a computing device. Such computerprogram product can be tangibly embodied in a carrier medium carryingmachine-readable code for execution by a programmable processor. Thepresent invention thus relates to a carrier medium carrying a computerprogram product that, when executed on computing means, providesinstructions for executing any of the methods as described above. Theterm “carrier medium” refers to any medium that participates inproviding instructions to a processor for execution. Such a medium maytake many forms, including but not limited to, non-volatile media, andtransmission media. Non volatile media includes, for example, optical ormagnetic disks, such as a storage device which is part of mass storage.Common forms of computer readable media include, a CD-ROM, a DVD, aflexible disk or floppy disk, a tape, a memory chip or cartridge or anyother medium from which a computer can read. Various forms of computerreadable media may be involved in carrying one or more sequences of oneor more instructions to a processor for execution. The computer programproduct can also be transmitted via a carrier wave in a network, such asa LAN, a WAN or the Internet. Transmission media can take the form ofacoustic or light waves, such as those generated during radio wave andinfrared data communications. Transmission media include coaxial cables,copper wire and fibre optics, including the wires that comprise a buswithin a computer.

By way of illustration, the present invention not being limited thereto,the example of a computerized system being an anthropomorphic armequipped with an under-actuated biomechatronic hand will be described inmore detail, as well as examples of methods for processing signals forcontrol of such an anthropomorphic arm equipped with a biomechatronichand and methods for controlling such a system. It is to be noticedthat, although system features and/or method features are described forthe anthropomorphic arm equipped with biomechatronic hand, these can beincorporated, applied and/or used in different computerised systems,methods for data processing in different computerised systems, orcontrol methods of different computerised systems, within the scope ofthe present invention. The examples illustrate particular features andsome of the advantages obtainable by embodiments according to thepresent invention

Example 1 Computerised System

A computerised system 600 as discussed in the present example is shownschematically in FIG. 11, wherein both the anthropomorphic arm 602 andthe biomechatronic hand 604 are indicated. The biomechatronic hand 604with two fingers mounted and sensorised was installed, whereby eachfinger was controlled by only one pulling cable, in such a way that thehand adaptively can wrap the grasped objects. A task controller 610 isused for controlling the arm 602 and/or hand 604 resulting in a smoothand vibrations-free movement. In the present example, the inner surfaceof each fingertip was covered with three tactile modules, consisting ofa flexible array of micro-fabricated force sensors 606 andelectro-optical converters. In this particular case, each module arrayconsists of a number of microfabricated triaxial force sensors 606assembled on a flexible substrate and covered with a soft polyurethanelayer with protection and grip purposes. Three electrical signals areextracted from each sensor and processed and converted into light pulsesby the opto-electronics embedded in each module. Light-coded tactileinformation form a tactile image is transferred to the data processor608, in the present example being a cellular neural network CNN visualmicroprocessor, hosted together with the task controller 610 in ageneral processor 612, in the present example being an industrial PC, aBi-I v2 PC from AnaLogic Computers Ltd., Budapest, Hungary. The taskcontroller 610 used was a Texas Instruments TMS320C6415 digital dataprocessor (DSP) running at 600 MHz, and the data processor 608 was anultra fast ACE16K CNN visual microprocessor (from AnaFocus Ltd6), beinga digitally-controlled analog array processor used as input device andmain processing unit (arithmetic processor). This processor consists ofa 128×128 CCD optical input layer on top of a 128×128 cells CNNarchitecture. Appropriate time information is obtained by appropriatelysetting an integration time (shutter) on the visual microprocessor, i.e.sampling, such that the acquired tactile data are obtained as dynamicgrey-scale images (brighter areas indicate stronger stimuli). Thepositions of sensors 606 and pixels in the image are topologicallyconsistent, i.e. the location of a given sensor 606 on the fingertipsurface corresponds to the relative position of the corresponding pixelsin the tactile image. In the present example, a single tactile channelprojects its information over a small group of more or less 16 adjacentpixels. A portion of the input image corresponds to thumb sensors, whileanother to index sensors. Every feature is always computed over thewhole input image, with no computational overload. In the presentexample, mask images are used to extract the positions where a givenfeature was detected, by means of CNN morphological and logicaloperations. The data processor 608 thus uses a CNN based processor,providing a given CNN-based feature extraction algorithm that returns abinary image in which white portions indicate areas of the image (and ofthe corresponding input space), where that feature has been detected. Asthree light signals, only poorly correlated with the stimulus direction,are extracted from each sensor 606, so that the exteroceptive systemused in this work consists of 54 tactile optical signals. A rigorouscalibration of the sensors 606 arrays would produce signals codifyingprecisely the input stimulus intensity along each direction, but theapproach followed here made this step unnecessary: the data processor608 is fed with signals only poorly correlated with the stimulusdirection; alternatively said, the implemented tactile system accepts asinput raw and direction-uncorrelated signals. In the present example, acable made of plastic optical fibres (POFs) convey optically codedinformation from the tactile sensors on the hand fingers to the generalprocessor 612, projecting the tactile image on the optical input layerof the ACE16K visual microprocessor and processing the imagesimmediately in the CNN architecture that is hosted below the opticallayer (each pixel is the optical input of the corresponding CNN cell).The system may comprise a coupling element for focussing the light ontothe chip. The result of the tactile image processing inside the CNNchip, which represents a quality of features detected or the correlatingsemantic representation thereof, is transferred to the task controller610 and used together with proprioceptive information received fromencoders on the arm and hand 614, for providing e.g. positionalinformation. The general processor 612 communicates with the arm 602 andthe hand 604 or their local controller, e.g. over a network. FIG. 11shows the different flows of information in the architecture of therobotic system used in the present example.

Example 2 Detection of Features

For the present example of grasping, algorithms based on the recognitionof three kinds of tactile events, recognized in the temporal domain areused, being variation, oscillation and vibration. These three types offeatures are, in the present description, described as follows:

-   -   variation: a given tactile signal contains a variation if its        intensity changes by at least a given threshold a in the same        period T. The threshold σ thereby is, in the present example,        defined in terms of pixel intensity, expressed in number of grey        levels (out of 255) and depends on the noise level of the        tactile signals; a variation can be positive, if the intensity        increases, or negative. A preliminary characterization of the        tactile module and the noise performances of the sensors in the        present example allowed to consider σ=5 grey levels (2% of the        entire dynamic range).    -   oscillation: an oscillation is the composition of two subsequent        variations of opposite sign in the same period T, with        amplitudes of a least n.σ and m.σ respectively, thus showing a        certain tendency to periodicity. In the present example, n=3,        m=1 proved to be a good choice to detect oscillations in normal        manipulation.    -   vibration: a vibration is the sequence of at least two        oscillations in two subsequent periods T.

FIG. 12 a schematically represents tactile signals as they areclassified in this work: the first and second signals 652, 654 arevariations but not oscillations, as the tendency to periodicity ismissing. In the third and fourth signals 656, 658 the two conditions foran oscillation to be detected are matched. The last curve 662 is asequence of at least two oscillations, and is recognized as vibrationswhile the fifth curve 660 does not match the condition on minimumamplitude of variations and is not recognized as vibration. It is worthnoting that, within the proposed naming convention, all signals 662,664, 666 shown in FIG. 12 a are considered as vibrations, even thoughsignal 664 is clearly non periodic. With these naming conventions, thedetection of tactile signals vibrations (often associated with slippage)can be reformulated as the detection of oscillations which occur insidetwo consecutive periods. As both amplitude and frequency of oscillationsappearing during slip may vary, depending on the properties of the skin,on the weight, material and softness of the object and, of course, onthe sensibility and capability of the sensors of detecting suddenlychanging stimuli, the oscillations with amplitude and frequency valueswere detected within a quite large range. In the present example,detection of tactile signals with frequency content within the range20-50 Hz was found appropriate. The period used in the present examplewas T=25 ms. With a sampling period t of 2.5 ms, a sampling rate of 400Hz was obtained. These are indicated in FIG. 12 b.

In the present example, the algorithm, running in parallel in the dataprocessor for all sampled sensor outputs, searches for signaloscillations within consecutive periods of T seconds. In order to copewith memory size concerns and minimisation of comparisons and updates,the time as divided into contiguous period windows, each of them ofduration T. In the present example, the drawbacks of selectingcontiguous period windows instead of e.g. a sliding window, did not havesignificant effects on the whole tactile information processing chain.Nevertheless, using larger computing power, also a sliding window couldbe selected.

In the present example, feature extraction is realised by parallelprocessing of all the tactile sensorial information while assuring ahigh frame rate for image capturing, resulting in a computationalefficient system. The algorithms used were designed so as to minimizethe number of data transfers to the task controller and were based onusing only stable and reliable CNN operators.

By way of illustration, FIG. 13 illustrates an exemplary block diagramof the general CNN algorithm 700 used for detecting tactile events inthe present example. In the initialisation phase 702 the image buffersare set to a proper initial black value (by means of the filblackoperator). Then the first image, labelled e.g. “firstperiodimage”, isacquired 704 and denoised on the ACE16k processor and it is stored asthe first tactile image of the examined period. A time delay is set 706,to synchronize the acquisition rate and then a following image, labellede.g. “currentimage”, is acquired 708. In step 710, the detection oftactile signals is performed, in the present example being detection ofvariations. The detection of these tactile signals is basically based onCNN operators to realize, at each iteration, a comparison between theimages ‘CurrentImage’ and ‘FirstPeriodImage’. An algorithm for suchdetection may have a symmetrical structure, thus allowing detection ofpositive and negative variations. The comparison may involve thesubtract and the threshold CNN operators, and aims at evaluating howmuch the tactile signals are varying with respect to the tactileconditions at the beginning of the current period. More precisely, bythresholding the subtractions images several binary images are obtained,whereby each of those indicate all the regions that have changed formore than a given threshold n.σ. The accuracy of such detection islimited by the value of the least reliably measurable variation σ. Thisprocedure results in the update of a set of binary images which storethe presence of variations of the tactile signals. In the general CNNalgorithm 700, the previous steps are repeated, with a new‘CurrentImage’ at each iteration, till the end of the period isdetermined in a decision step 712 as indicated in FIG. 13. At the end ofeach period, a cross-check on the binary images is performed in order toisolate the tactile areas where oscillations occurred during the lastperiod, as indicated in step 714. A more detailed overview of anexemplary algorithm 725 for this detection is shown in FIG. 14. Couplesof the binary images are compared by means of the logAND CNN operator,being a logical operator. Each AND is employed to detect oscillationswith a particular wave shape along the examined period. Further, theimages are coupled, at the input of the AND operators, in such a way toensure that the oscillations spread over a sufficiently wide luminanceamplitude. The logOR operator is used to collect the tactile signalsthat have oscillated along the examined period in only one image named‘OscImage’. Finally, vibrations are detected, as indicated in step 716,in the areas where oscillations occurred in the last two periods. Todetect vibrations, for example the images ‘OscImage’ obtained in theprevious step and associated to two consecutive periods are ANDed(logAND operator). After that, the general CNN algorithm 700 performs arefresh of some images, as indicated in step 718, so as to set up thevariables state to perform the analysis of the next time period. Inparticular the last tactile image of the last period becomes the firstand reference image of the next period (FirstPeriodImage=CurrentImage).

In order to prevent errors caused by defects in the system, the processmay be initiated with a warming phase, wherein the appropriate operationof the sensors is checked. An example of an algorithm 730 that may beused for this is shown in FIG. 15. When the hand is open, a sequence ofimages is acquired, as indicated in a first step 732, and fed into aprocedure that performs a sensors check, indicated in step 734. Thesensor check procedure evaluates all image areas and indicates thoseimages that are activated 735 or flickering 737, and thus not black overthe entire period, as broken sensors. The latter can be decided as forthe open hand, no tactile signal should be sensed by the sensors. Whenit is decided that a sufficient check is performed, indicated bydecision step 736, an image of trustable areas 739 is generated, thegeneration step 738 taking into account the non-trustable areasindicated during the sensor check step by excluding them from thecomputation of trustable areas. This feature increases the robustness ofthe whole tactile system, especially when many sensors are present andredundancy can, as in this case, be exploited. As shown in FIG. 15, theoutput of this procedure is a binary image (a mask in CNN language)containing the trustable areas.

Example 3 Determination of the Quality of Features

Determination of the quality of the features and correlating it withsemantic features in the present example is performed by checking thestability over time of the extracted features. The latter preferablytakes into account not only time features, as used for extractingfeatures from the data signal, but also spatial and intensity features.Such a mix may be especially suitable for grasping objects of extremelyslippery nature and having a smooth surface, such as e.g. with tofu.When grasping less slippery objects, in some cases only temporalinformation features is sufficient. Therefore, the safety of the graspmay also be based on a size of the contact area and on the minimumapplied force control.

Determination of the quality of features, may e.g. be performed as soonas some initial contact has been detected. By way of illustration, thepresent invention not being limited thereto, an exemplary CNN algorithm,as can be used is shown in FIG. 16. Determination of the quality offeatures in the present example includes determining the stability overtime of the extracted features. The exemplary algorithm 750 performs acheck on the stability over time of the trustable areas of the incomingtactile images, by means of gray-scale and morphological operationscorresponding with receiving an input sequence of images in step 752 anddetecting vibration 754 based on trustable area information 753. Inparticular, analog differences between subsequent images and binarythresholding operations are used. In order to keep the computationalrequirements low, a single binary image is used to keep trace of thevarying parts of the tactile input sequence. A binary image containingstable active areas is finally generated, shown by step 755. A thresholdis then applied on these areas input image on the locationscorresponding to active areas, to assure that the safe contact area issufficiently large. As two different sensory channels project on twogeometrically separated (non connected) areas of the tactile image, alsocalled blobs, this phase can be described, for the sake of simplicity,as a count of the active blobs, illustrated by step 756. A decision step758 then allows to determine whether the contact is safe, indicated bystep 760 or not safe, indicated by step 762. In case the safe contactarea is not large enough on one or both fingers, or the intensity of thetactile signal on stable areas is not sufficient, the procedure willtrigger a small contracting pulse on the corresponding finger motors.The friction coefficient between the finger and the object is notestimated with this approach to contact quality assessment, ant theentire procedure, with a duration of 100 ms, is carried on beforelifting the arm. The computation involves therefore 40 images acquiredat 400 Hz.

Example 4 Task Control

Using for example the above specified algorithms, the invention notbeing limited thereto, the task of grasping and lifting is discussedfurther in the present example, with reference to the computerisedsystem as described in example 1. The method 800 for performing the taskconsists of the following phases, described with reference to FIG. 17where part of a finite state modelling of the task is shown.

In a first step, i.e. step 802, a warming phase is performed wherein anumber of images are acquired with the hand open, and a check isperformed to find broken sensors and exclude them from followingprocessing. Such warming phase may be performed using an algorithm asdescribed in FIG. 15, although the invention is not limited thereto.After an initial warming phase during which the system isolates theeventually broken sensors, the robotic hand is expected to close aroundthe object (hand closing phase) until contact is detected. In a secondstep 804, the grasping therefore is initiated by closing the fingers ofthe hand independently actuated. These close until some contact with anobject is detected. The detection of some contact is the only signalable of stopping the hand closure. The contact event triggers themigration to the Stability check state, which evaluates the quality ofthe contact. In a third step 806, the quality of the contact thus isevaluated to assure a suitable grasp before lifting the hand. Asufficient grasp quality is eventually reached by means of small andquick closing pulses on the fingers until certain criteria on the inputtactile image are satisfied. In case of insufficient stability, the handmay perform a small but sudden additional closure and if no sufficientstability is obtained, the system may indicate a fail, as indicated bystep 808 and e.g. open the hand. After a stable grasp has been reached,in a fourth step 810, the hand lifts the object, reacting to tactileevents like vibrations, or oscillations such those arising during slipor external stimulation, through small and quick closing pulses. Thelifting in the present example is performed by lifting the arm until aheight of 10 cm is reached. In an optional fifth step 812, a furthermanipulation such as e.g. movement or stand still of the object isinduced. In the present example, the object was held in place for tenseconds. The latter is to illustrate the stability of the method. Duringboth this and the previous step, the controller causes the hand to closeby small and quick movements in case slip is detected. In a sixth step814, the object is released and the arm is returned to the initialposition.

The above method is controlled by the task controller. Transitionsbetween the different tasks may be regulated either by proprioceptiveinformation such as positional information from arm or hand encoders orsuch as time intervals from a timer or by exteroceptive semanticinformation, obtained from the peripheral part, i.e. the sensors, fromthe computerised system.

By way of illustration, further examples are provided whereby movementor intention of movement are detected. It is to be noticed that theseexamples are not to be considered as limiting. The examples illustratesdetection of a mechanical stimulus (movement) in an optical way. In theexamples the movement of an object in a traffic situation is derivedusing optical detectors. The movement is derived in similar way asdescribed above, i.e. extraction of features with dynamic behavior isperformed taking into account topology of the sensor outputs. Suchfeatures may for example be local changes like oscillations orvibrations within a pattern or part thereof.

Example 5

In the fifth example, detection of the intention of a pedestriancrossing a road is illustrated. In a first step, the pedestrian may bedetected by conventional detection means, as known by the person skilledin the art. Different algorithms for the detection and identification ofa head and body of a pedestrian are known and can be implemented in asystem according to the present invention, e.g. by operating on an imageof the traffic scene detected with the system. For example, the body andhead of a pedestrian may be represented as a cylinder and ellipserespectively. Recognition of pedestrians and the algorithms applied, maybe performed in combination with the system according to the invention.In this way, a simplified model of a pedestrian may be projected asinput. Processing of the image for extracting features with a dynamicbehaviour then may be performed using a system as illustrated above forthe grasping movement. The latter may be used for recognising avibration of the head with respect to the body, representing themovement that a pedestrian makes by looking left and right beforecrossing the road. Such a vibration can in this case for example bedetected by detecting a vibration of a head or a representationtherefore on a non oscillating body or a representation thereof. Thesame algorithms can be applied as provided in the above describedexamples, wherein the event detected, being the crossing of a road by apedestrian or the intention of crossing of a road by a pedestrian, isbased on the recognition of features with a dynamic behaviour, being theoscillation of the head. The event thus can be recognised by detectionof spatial and temporal features having a dynamic behaviour. It is anadvantage of embodiments according to the present invention that, by theparallel processing, the technology makes it possible to detect multiplepedestrians present on the scene, without additional computationalrequirements, while keeping trace of where the stimuli arose. FIG. 18illustrates a model for the pedestrian, whereby an oscillating motion ofa head with respect to the body could be identified as an indication ofan event wherein the pedestrian will cross a road. A system, e.g.vehicle, may be provided with a task controller for controlling a taskto be performed, a set of sensors, a sensor output processing means anda quality determining system for determining a quality of a feature orevent occurring may be provided. The sensor output processing means andthe quality determining system may be as set out in examples 2 and 3.Detection of an oscillation may be detection of an oscillation in thesignal representative for the outer portions of the head or the modelthereof. The sensor arrays may be optical detectors for opticallydetecting a movement.

The task controller may be responsive to the obtained quality of thefeature or event detected and may upon e.g. detection of a pedestriancrossing the road initiate a vehicle to slow down or stop. Other actionsaccording to predetermined rules or programmed or derived algorithmsalso may be performed.

Example 6

Movement of a vehicle in willing to change lane or to start a take overmaneuver is another example where embodiments according to the presentinvention can be applied. Detection or recognition of vehicles in animage can be performed by conventional techniques, as known by theperson skilled in the art. Such methods can be easily combined withmethods and systems according to the present invention. The image orobtained model can be projected, i.e. after an optional pre-processingstage, and obtained for processing. Such projection may be providing anoptical input to the processor, if e.g. a processor with optical inputas described in some of the embodiments above is used. The plurality ofsensors used may be a plurality of optical sensors. The processing meansmay be as set out in the previous examples, as well as the qualitydetermination means. A vehicle approaching or moving away will appear asstatic, while for a vehicle starting to perform a maneuver or changelane, features having dynamic behavior such as variation and oscillationcan be detected by spatially and temporally filtering a plurality ofparallel processed temporal sensor outputs. The methods and systems asset out in embodiments above thus can be applied for detecting events.Oscillation and/or vibration of parts of the vehicle thus can bederived. By way of illustration, FIG. 19 illustrates a schematicrepresentation of a vehicle whereby oscillation of the vehicle isillustrated, as can be detected using methods and systems according toembodiments of the present invention. A stable centre position as wellas an outer left and outer right position of the vehicle is indicated.

It is to be understood that although preferred embodiments, specificconstructions and configurations, as well as materials, have beendiscussed herein for devices according to the present invention, variouschanges or modifications in form and detail may be made. For example,any formulas given above are merely representative of procedures thatmay be used. Functionality may be added or deleted from the blockdiagrams and operations may be interchanged among functional blocks.Steps may be added or deleted to methods described within the scope ofthe present invention.

1-37. (canceled)
 38. A method (400) of processing signal outputs of aplurality of topologically distinct sensors in response to mechanicalstimuli, the method comprising obtaining (402) a plurality oftopologically arranged temporal sensor outputs in parallel, extractingfeatures (406) by arithmetic processing in parallel of neighbouringtemporal sensor outputs in a topology consistent way, the features beingidentified by a dynamic behaviour pattern, and a quality of theextracted features being determined.
 39. A method (400) according toclaim 38, wherein extracting features taking in a topology consistentway comprises arranging the features in a topology maintaining array orarrays (408).
 40. A method (400) according to claim 38, whereinobtaining (402) a plurality of temporal sensor outputs in parallelcomprises sampling (404) a plurality of sensor outputs in time togenerate the temporal sensor outputs.
 41. A method (400) according toclaim 38, wherein the temporal sensor outputs are sensor outputs frommechanical sensors or tactile sensors.
 42. A method (400) according toclaim 38, wherein arithmetic processing comprises processing saidneighbouring temporal sensor outputs in the temporal domain.
 43. Amethod (400) according to claim 38, wherein said extracting of featuresin a topology consistent way comprises recognising the features insensor outputs and preserving their spatial localisation.
 44. A method(400) according to claim 39, the method further comprising spatiallyfiltering (410) or spatio-temporal filtering of the array or arrays. 45.A method (400) according to claim 44, wherein said spatially filtering(410) comprises extraction of an event based on a plurality ofneighbouring extracted features in said topology maintaining array orarrays.
 46. A method (400) according to claim 38, wherein the arithmeticprocessing comprises applying a bandpass, low-pass or high-pass filteror a combination of bandpass, low-pass or high-pass filters for temporalfeature extraction, and/or wherein the arithmetic processing comprisesusing a cellular non-linear network, and/or wherein the arithmeticprocessing comprises only adding, subtracting, delaying, thresholding,logical or morphological functions, and/or wherein the arithmeticprocessing comprises applying a filter having a single filter inputsignal and resulting in multiple filter output signal and wherein eachfilter output signal is based on the subtraction of a modified inputsignal from an original input signal and/or wherein arithmeticprocessing in parallel of neighbouring temporal sensor outputs comprisesarithmetic processing in parallel of only part of the neighbouringtemporal sensor outputs.
 47. A method (400) according to claim 38,wherein the dynamic behaviour comprises any of a variation (606, 608),vibration and/or oscillation (610, 612, 614, 616).
 48. A method (400)according to claim 38, wherein said quality of the extracted featurescomprises a stability of the dynamic behaviour among a plurality ofneighbours.
 49. A method according to claim 38, wherein the method isadapted for processing said sensor outputs for improving active orpassive perception of events or features occurring in the environment.50. A method for controlling a computerised system comprising at leastone movable element comprising a plurality of topologically distinctsensors, the method comprising measuring a plurality of output signalsfrom said mechanical sensors, processing said plurality of outputsignals using a method (400) according to claim 38, and controlling saidat least one movable element based on said determined quality of theextracted features.
 51. A data processor (100) for processing signaloutputs of a plurality of topologically distinct sensors in response tomechanical stimuli, the processor comprising an input means (102) forreceiving a plurality of topologically arranged temporal sensor outputsin parallel, a feature extracting means (106) for extracting featureswith a dynamical behaviour pattern in a topology consistent way, thefeature extracting means (106) comprising an arithmetic processor (110)for arithmetic processing in parallel neighbouring temporal samples ofthe sensor outputs, and a determination means (108) for determining aquality of the extracted features.
 52. A data processor (100) accordingto claim 51, wherein the input means (102) for receiving a plurality oftemporal sensor outputs in parallel comprises a sampling means (104) forsampling a plurality of sensor outputs in time to generate the temporalsensor outputs.
 53. A data processor (100) according to claim 51,wherein the arithmetic processor (110) is adapted for arithmeticprocessing of said neighbouring temporal sensor outputs in the temporaldomain.
 54. A data processor (100) according to claim 51, the dataprocessor (100) furthermore being adapted for arranging the extractedfeatures in a topology maintaining array or arrays or the data processorbeing adapted for recognising features in sensor outputs and preservingtheir spatial localisation, or the data processor (100) furthermorebeing adapted for arranging the extracted features in a topologymaintaining array or arrays and comprising a filter for spatially orspatio-temporal filtering of the array or arrays.
 55. A data processor(100) according to claim 54, the filter comprising filtering means forextraction of an event based on a plurality of neighbouring extractedfeatures in said topology maintaining array or arrays.
 56. A dataprocessor (100) according to claim 51, wherein the arithmetic processor(110) comprises a bandpass filter or a combination of bandpass filtersfor extracting temporal features, and/or wherein the arithmeticprocessor (110) comprises a cellular neural network or a cellularnon-linear network, and/or wherein the arithmetic processor (110)comprises a filter based on only substracting, delaying, thresholding,logical or morphological functions, and/or wherein the arithmeticprocessor (110) comprises a filter having a single filter input signaland resulting in multiple filter output signal and wherein each filteroutput signal is based on the subtraction of a modified input signalfrom an original input signal, and/or wherein, the arithmetic processor(110) for processing in parallel neighbouring temporal sensor outputs isan arithmetic processor (110) for arithmetic processing in parallel onlypart of the neighbouring temporal sensor outputs.
 57. A data processor(100) according to claim 51, wherein the dynamic behaviour comprises anyof a variation (606, 608), vibration and/or oscillation (610, 612, 614,616).
 58. A data processor (100) according to claim 51, wherein saidquality of the extracted features comprises a stability of the dynamicbehaviour among a plurality of neighbours.
 59. A computerised system(300) comprising at least one movable element (302) comprising aplurality of topologically distinct sensors (20), and a data processor(100) according to claim
 51. 60. A computerised system (300) accordingto claim 59, the system further comprising a controller (30) forcontrolling the at least one moveable element as function of theobtained quality of the extracted features.
 61. A computer programproduct for processing signal outputs of a plurality of topologicallydistinct mechanical sensors in response to mechanical stimuli, thecomputer program product, when executed on a computer, adapted forexecuting the method according to claim
 38. 62. A machine-readable datastorage device non-transitorily storing the computer program product ofclaim 61.